<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
  <meta charset="utf-8" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />

  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>RockPaperScissorsTrainer &mdash; Melodie 0.6.0 documentation</title>
      <link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
      <link rel="stylesheet" href="../_static/css/theme.css" type="text/css" />
  <!--[if lt IE 9]>
    <script src="../_static/js/html5shiv.min.js"></script>
  <![endif]-->
  
        <script data-url_root="../" id="documentation_options" src="../_static/documentation_options.js"></script>
        <script src="../_static/jquery.js"></script>
        <script src="../_static/underscore.js"></script>
        <script src="../_static/_sphinx_javascript_frameworks_compat.js"></script>
        <script src="../_static/doctools.js"></script>
        <script async="async" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
    <script src="../_static/js/theme.js"></script>
    <link rel="author" title="About these documents" href="../about.html" />
    <link rel="index" title="Index" href="../genindex.html" />
    <link rel="search" title="Search" href="../search.html" />
    <link rel="next" title="CovidContagionVisual" href="covid_contagion_visual.html" />
    <link rel="prev" title="CovidContagionCalibrator" href="covid_contagion_calibrator.html" /> 
</head>

<body class="wy-body-for-nav"> 
  <div class="wy-grid-for-nav">
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search" >
            <a href="../index.html" class="icon icon-home"> Melodie
            <img src="../_static/logo.jpg" class="logo" alt="Logo"/>
          </a>
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../search.html" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>
        </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
              <ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../installation.html">Installation</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../installation.html#user-installation">User Installation</a></li>
<li class="toctree-l2"><a class="reference internal" href="../installation.html#developer-installation">Developer Installation</a></li>
<li class="toctree-l2"><a class="reference internal" href="../installation.html#dependency-note">Dependency Note</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../introduction.html">Introduction</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../introduction.html#agent-based-model-abm">Agent-based Model (ABM)</a></li>
<li class="toctree-l2"><a class="reference internal" href="../introduction.html#melodie-framework">Melodie Framework</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../introduction.html#model">Model</a></li>
<li class="toctree-l3"><a class="reference internal" href="../introduction.html#scenario">Scenario</a></li>
<li class="toctree-l3"><a class="reference internal" href="../introduction.html#modelling-manager">Modelling Manager</a></li>
<li class="toctree-l3"><a class="reference internal" href="../introduction.html#infrastructure">Infrastructure</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../tutorial.html">Tutorial</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../tutorial.html#project-structure">Project Structure</a></li>
<li class="toctree-l2"><a class="reference internal" href="../tutorial.html#agent">Agent</a></li>
<li class="toctree-l2"><a class="reference internal" href="../tutorial.html#scenario">Scenario</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../tutorial.html#generate-agent-params">Generate <code class="docutils literal notranslate"><span class="pre">agent_params</span></code></a></li>
<li class="toctree-l3"><a class="reference internal" href="../tutorial.html#load-simulator-scenarios">Load <code class="docutils literal notranslate"><span class="pre">simulator_scenarios</span></code></a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../tutorial.html#model">Model</a></li>
<li class="toctree-l2"><a class="reference internal" href="../tutorial.html#environment">Environment</a></li>
<li class="toctree-l2"><a class="reference internal" href="../tutorial.html#datacollector">DataCollector</a></li>
<li class="toctree-l2"><a class="reference internal" href="../tutorial.html#last-words">Last Words</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../framework_comparison.html">Framework Comparison</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../framework_comparison.html#project-structure">Project Structure</a></li>
<li class="toctree-l2"><a class="reference internal" href="../framework_comparison.html#model-components">Model Components</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../framework_comparison.html#melodie">Melodie</a></li>
<li class="toctree-l3"><a class="reference internal" href="../framework_comparison.html#id2">Mesa</a></li>
<li class="toctree-l3"><a class="reference internal" href="../framework_comparison.html#id3">AgentPy</a></li>
<li class="toctree-l3"><a class="reference internal" href="../framework_comparison.html#summary">Summary</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../framework_comparison.html#scenario-management">Scenario Management</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../framework_comparison.html#id4">Melodie</a></li>
<li class="toctree-l3"><a class="reference internal" href="../framework_comparison.html#id5">Mesa</a></li>
<li class="toctree-l3"><a class="reference internal" href="../framework_comparison.html#id6">AgentPy</a></li>
<li class="toctree-l3"><a class="reference internal" href="../framework_comparison.html#id7">Summary</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../framework_comparison.html#modeling-manager">Modeling Manager</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../framework_comparison.html#calibrator">Calibrator</a></li>
<li class="toctree-l3"><a class="reference internal" href="../framework_comparison.html#trainer">Trainer</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1 current"><a class="reference internal" href="_index.html">Model Gallery</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="covid_grid_contagion.html">CovidGridContagion</a><ul>
<li class="toctree-l3"><a class="reference internal" href="covid_grid_contagion.html#project-structure">Project Structure</a></li>
<li class="toctree-l3"><a class="reference internal" href="covid_grid_contagion.html#grid-and-spot">Grid and Spot</a></li>
<li class="toctree-l3"><a class="reference internal" href="covid_grid_contagion.html#matrix-data">Matrix Data</a></li>
<li class="toctree-l3"><a class="reference internal" href="covid_grid_contagion.html#model">Model</a></li>
<li class="toctree-l3"><a class="reference internal" href="covid_grid_contagion.html#gridagent">GridAgent</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="covid_network_contagion.html">CovidNetworkContagion</a><ul>
<li class="toctree-l3"><a class="reference internal" href="covid_network_contagion.html#project-structure">Project Structure</a></li>
<li class="toctree-l3"><a class="reference internal" href="covid_network_contagion.html#model">Model</a></li>
<li class="toctree-l3"><a class="reference internal" href="covid_network_contagion.html#scenario">Scenario</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="covid_contagion_calibrator.html">CovidContagionCalibrator</a><ul>
<li class="toctree-l3"><a class="reference internal" href="covid_contagion_calibrator.html#project-structure">Project Structure</a></li>
<li class="toctree-l3"><a class="reference internal" href="covid_contagion_calibrator.html#calibrator">Calibrator</a></li>
<li class="toctree-l3"><a class="reference internal" href="covid_contagion_calibrator.html#algorithm">Algorithm</a></li>
<li class="toctree-l3"><a class="reference internal" href="covid_contagion_calibrator.html#results">Results</a></li>
</ul>
</li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">RockPaperScissorsTrainer</a><ul>
<li class="toctree-l3"><a class="reference internal" href="#model-setup">Model Setup</a></li>
<li class="toctree-l3"><a class="reference internal" href="#trainer">Trainer</a></li>
<li class="toctree-l3"><a class="reference internal" href="#results">Results</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="covid_contagion_visual.html">CovidContagionVisual</a><ul>
<li class="toctree-l3"><a class="reference internal" href="covid_contagion_visual.html#visualizer">Visualizer</a></li>
<li class="toctree-l3"><a class="reference internal" href="covid_contagion_visual.html#melodiestudio">MelodieStudio</a></li>
<li class="toctree-l3"><a class="reference internal" href="covid_contagion_visual.html#how-to-start">How to start?</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../advanced/_index.html">Advanced Topics</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../advanced/troubleshooting.html">TroubleShooting</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../advanced/troubleshooting.html#port-already-in-use">Port Already in use</a><ul>
<li class="toctree-l4"><a class="reference internal" href="../advanced/troubleshooting.html#windows">Windows</a></li>
<li class="toctree-l4"><a class="reference internal" href="../advanced/troubleshooting.html#macos-linux-or-other-nix-systems">MacOS, Linux or other *nix systems</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../advanced/melodie_error_list.html">Melodie Errors</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../api/_index.html">API Reference</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../api/model.html">Model</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/agent.html">Agent</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/agent_list.html">AgentList</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/environment.html">Environment</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/data_collector.html">DataCollector</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/grid.html">Grid</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/network.html">Network</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/data_info.html">DataInfo</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../api/data_info.html#dataframeinfo">DataFrameInfo</a></li>
<li class="toctree-l3"><a class="reference internal" href="../api/data_info.html#matrixinfo">MatrixInfo</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../api/data_loader.html">DataLoader</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/scenario.html">Scenario</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/simulator.html">Simulator</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/calibrator.html">Calibrator</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/trainer.html">Trainer</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/db.html">DB</a></li>
<li class="toctree-l2"><a class="reference internal" href="../api/exceptions.html">Melodie Exceptions</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../changelog.html">Change Log</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../changelog.html#major-version-0-x">Major Version 0.x</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../changelog.html#v0-6-0-jan-04-2023">v0.6.0 (Jan. 04, 2023)</a></li>
<li class="toctree-l3"><a class="reference internal" href="../changelog.html#v0-5-0-dec-17-2022">v0.5.0 (Dec. 17, 2022)</a></li>
<li class="toctree-l3"><a class="reference internal" href="../changelog.html#v0-4-2-dec-15-2022">v0.4.2 (Dec. 15, 2022)</a></li>
<li class="toctree-l3"><a class="reference internal" href="../changelog.html#v0-4-1-dec-12-2022">v0.4.1 (Dec. 12, 2022)</a></li>
<li class="toctree-l3"><a class="reference internal" href="../changelog.html#v0-4-0-nov-15-2022">v0.4.0 (Nov. 15, 2022)</a></li>
<li class="toctree-l3"><a class="reference internal" href="../changelog.html#v0-3-0-oct-28-2022">v0.3.0 (Oct. 28, 2022)</a></li>
<li class="toctree-l3"><a class="reference internal" href="../changelog.html#v0-2-0-oct-24-2022">v0.2.0 (Oct. 24, 2022)</a></li>
<li class="toctree-l3"><a class="reference internal" href="../changelog.html#v0-1-1-aug-23-2022">v0.1.1 (Aug. 23, 2022)</a></li>
<li class="toctree-l3"><a class="reference internal" href="../changelog.html#v0-1-0-jul-22-2022">v0.1.0 (Jul. 22, 2022)</a></li>
<li class="toctree-l3"><a class="reference internal" href="../changelog.html#start-may-10-2021">Start (May. 10, 2021)</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../contribution.html">Contribution</a><ul>
<li class="toctree-l2"><a class="reference internal" href="../contribution.html#report-bugs">Report Bugs</a><ul>
<li class="toctree-l3"><a class="reference internal" href="../contribution.html#report-melodie-bugs">Report Melodie Bugs</a></li>
<li class="toctree-l3"><a class="reference internal" href="../contribution.html#report-documentation-bugs">Report Documentation Bugs</a></li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../contribution.html#ask-for-functionalities">Ask for Functionalities</a></li>
<li class="toctree-l2"><a class="reference internal" href="../contribution.html#how-to-contribute">How to Contribute</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../about.html">About</a></li>
</ul>

        </div>
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../index.html">Melodie</a>
      </nav>

      <div class="wy-nav-content">
        <div class="rst-content">
          <div role="navigation" aria-label="Page navigation">
  <ul class="wy-breadcrumbs">
      <li><a href="../index.html" class="icon icon-home"></a> &raquo;</li>
          <li><a href="_index.html">Model Gallery</a> &raquo;</li>
      <li>RockPaperScissorsTrainer</li>
      <li class="wy-breadcrumbs-aside">
            <a href="../_sources/gallery/rock_paper_scissors.rst.txt" rel="nofollow"> View page source</a>
      </li>
  </ul>
  <hr/>
</div>
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
             
  <section id="rockpaperscissorstrainer">
<h1>RockPaperScissorsTrainer<a class="headerlink" href="#rockpaperscissorstrainer" title="Permalink to this heading"></a></h1>
<p>To show how the <code class="docutils literal notranslate"><span class="pre">Trainer</span></code> module can be used,
we provide this <a class="reference external" href="https://github.com/ABM4ALL/RockPaperScissorsTrainer">RockPaperScissorsTrainer</a> model,
which implements the “evolutionary training” framework proposed in this
<a class="reference external" href="https://ieeexplore.ieee.org/document/9857838/">paper</a>:</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Yu, S. (2022). An Agent-Based Framework for Policy Simulation: Modeling Heterogeneous Behaviors With Modified Sigmoid Function and Evolutionary Training. <em>IEEE Transactions on Computational Social Systems</em>.</p>
</div>
<p>The <code class="docutils literal notranslate"><span class="pre">RockPaperScissorsTrainer</span></code> model is not as sophisticated as the one in the paper,
as the agents’ behaviors are not modeled with the modified sigmoid function,
however, the evolutionary training is implemented based on the
<a class="reference external" href="https://en.wikipedia.org/wiki/Genetic_algorithm">genetic algorithm (GA)</a> in the same way.</p>
<p>Before diving in, we will strongly suggest you to read the <a class="reference internal" href="../tutorial.html#tutorial"><span class="std std-ref">Tutorial</span></a> section and get familiar with the
<a class="reference external" href="https://github.com/ABM4ALL/CovidContagion">CovidContagion</a> model first,
as this <code class="docutils literal notranslate"><span class="pre">RockPaperScissorsTrainer</span></code> model follows similar structure.
The main difference is, you can run the model with either
(1) <code class="docutils literal notranslate"><span class="pre">Simulator</span></code> based on randomly initialized behavioral parameters of the agents, or
(2) <code class="docutils literal notranslate"><span class="pre">Trainer</span></code> so that the agents will individual update their behavioral parameters for higher payoff.</p>
<p>Then, you can load the trained behavioral parameters into the model to initialize the agents,
and then run the model again with the <code class="docutils literal notranslate"><span class="pre">Simulator</span></code>.
As explained in the <a class="reference external" href="https://ieeexplore.ieee.org/document/9857838/">paper</a> above,
by combining the optimization (evolutionary training) and simulation methods in such a framework,
the validity and flexibility of agent-based modeling are balanced.</p>
<section id="model-setup">
<h2>Model Setup<a class="headerlink" href="#model-setup" title="Permalink to this heading"></a></h2>
<p>In this model, the agents play the rock-paper-scissors game with each other.</p>
<p>We make following assumptions:</p>
<ul class="simple">
<li><p>First, winning, losing and tie results bring positive, negative, and 0 payoff to the agents.</p></li>
<li><p>Second, the agents have heterogeneous “preference”, i.e., wining or losing with different actions bring different payoff to the agents.</p></li>
</ul>
<p>As a result, theoretically, the agents should have different “behavioral parameters”.
For example, for a agent who gets higher payoff by winning with rock than paper,
it should have a higher probability to play rock than paper.</p>
<p>To model the “selection probability” of each option,
we assign three behavioral parameters to each agent: <span class="math notranslate nohighlight">\(\alpha_r\)</span>, <span class="math notranslate nohighlight">\(\alpha_p\)</span>, <span class="math notranslate nohighlight">\(\alpha_s\)</span>.
Then, the probability for the agent to play the three options are:</p>
<ul class="simple">
<li><p>Rock <span class="math notranslate nohighlight">\(\rightarrow\)</span> <span class="math notranslate nohighlight">\(\alpha_r / (\alpha_r + \alpha_p + \alpha_s)\)</span></p></li>
<li><p>Paper <span class="math notranslate nohighlight">\(\rightarrow\)</span> <span class="math notranslate nohighlight">\(\alpha_p / (\alpha_r + \alpha_p + \alpha_s)\)</span></p></li>
<li><p>Scissors <span class="math notranslate nohighlight">\(\rightarrow\)</span> <span class="math notranslate nohighlight">\(\alpha_s / (\alpha_r + \alpha_p + \alpha_s)\)</span></p></li>
</ul>
<p>To train the three behavioral parameters for each agent before producing meaningful model results with <code class="docutils literal notranslate"><span class="pre">Simulator</span></code>,
<code class="docutils literal notranslate"><span class="pre">Melodie</span></code> provides the <code class="docutils literal notranslate"><span class="pre">Trainer</span></code> module.</p>
</section>
<section id="trainer">
<h2>Trainer<a class="headerlink" href="#trainer" title="Permalink to this heading"></a></h2>
<p>Following the methodology introduced in the <a class="reference external" href="https://ieeexplore.ieee.org/document/9857838/">paper</a> above,
the <code class="docutils literal notranslate"><span class="pre">Trainer</span></code> module applies individual GA-based evolutionary training to “calibrate” the behavioral parameters of agents.
All the relevant mechanisms are written in the <code class="docutils literal notranslate"><span class="pre">trainer.py</span></code> file as follows.</p>
<div class="literal-block-wrapper docutils container" id="id5">
<div class="code-block-caption"><span class="caption-text">trainer.py</span><a class="headerlink" href="#id5" title="Permalink to this code"></a></div>
<div class="highlight-Python notranslate"><div class="highlight"><pre><span></span><span class="linenos"> 1</span><span class="kn">from</span> <span class="nn">Melodie</span> <span class="kn">import</span> <span class="n">Trainer</span>
<span class="linenos"> 2</span><span class="kn">from</span> <span class="nn">source.agent</span> <span class="kn">import</span> <span class="n">RPSAgent</span>
<span class="linenos"> 3</span>
<span class="linenos"> 4</span>
<span class="linenos"> 5</span><span class="k">class</span> <span class="nc">RPSTrainer</span><span class="p">(</span><span class="n">Trainer</span><span class="p">):</span>
<span class="linenos"> 6</span>
<span class="linenos"> 7</span>    <span class="k">def</span> <span class="nf">setup</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="linenos"> 8</span>        <span class="bp">self</span><span class="o">.</span><span class="n">add_agent_training_property</span><span class="p">(</span>
<span class="hll"><span class="linenos"> 9</span>            <span class="s2">&quot;agents&quot;</span><span class="p">,</span>
</span><span class="hll"><span class="linenos">10</span>            <span class="p">[</span>
</span><span class="hll"><span class="linenos">11</span>                <span class="s2">&quot;strategy_param_1&quot;</span><span class="p">,</span>
</span><span class="hll"><span class="linenos">12</span>                <span class="s2">&quot;strategy_param_2&quot;</span><span class="p">,</span>
</span><span class="hll"><span class="linenos">13</span>                <span class="s2">&quot;strategy_param_3&quot;</span>
</span><span class="hll"><span class="linenos">14</span>            <span class="p">],</span>
</span><span class="hll"><span class="linenos">15</span>            <span class="k">lambda</span> <span class="n">scenario</span><span class="p">:</span> <span class="nb">list</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="n">scenario</span><span class="o">.</span><span class="n">agent_num</span><span class="p">)),</span>
</span><span class="linenos">16</span>        <span class="p">)</span>
<span class="linenos">17</span>
<span class="linenos">18</span>    <span class="k">def</span> <span class="nf">collect_data</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="linenos">19</span>        <span class="bp">self</span><span class="o">.</span><span class="n">add_agent_property</span><span class="p">(</span><span class="s2">&quot;agents&quot;</span><span class="p">,</span> <span class="s2">&quot;strategy_param_1&quot;</span><span class="p">)</span>
<span class="linenos">20</span>        <span class="bp">self</span><span class="o">.</span><span class="n">add_agent_property</span><span class="p">(</span><span class="s2">&quot;agents&quot;</span><span class="p">,</span> <span class="s2">&quot;strategy_param_2&quot;</span><span class="p">)</span>
<span class="linenos">21</span>        <span class="bp">self</span><span class="o">.</span><span class="n">add_agent_property</span><span class="p">(</span><span class="s2">&quot;agents&quot;</span><span class="p">,</span> <span class="s2">&quot;strategy_param_3&quot;</span><span class="p">)</span>
<span class="linenos">22</span>        <span class="bp">self</span><span class="o">.</span><span class="n">add_agent_property</span><span class="p">(</span><span class="s2">&quot;agents&quot;</span><span class="p">,</span> <span class="s2">&quot;share_rock&quot;</span><span class="p">)</span>
<span class="linenos">23</span>        <span class="bp">self</span><span class="o">.</span><span class="n">add_agent_property</span><span class="p">(</span><span class="s2">&quot;agents&quot;</span><span class="p">,</span> <span class="s2">&quot;share_paper&quot;</span><span class="p">)</span>
<span class="linenos">24</span>        <span class="bp">self</span><span class="o">.</span><span class="n">add_agent_property</span><span class="p">(</span><span class="s2">&quot;agents&quot;</span><span class="p">,</span> <span class="s2">&quot;share_scissors&quot;</span><span class="p">)</span>
<span class="linenos">25</span>        <span class="bp">self</span><span class="o">.</span><span class="n">add_environment_property</span><span class="p">(</span><span class="s2">&quot;total_accumulated_payoff&quot;</span><span class="p">)</span>
<span class="linenos">26</span>
<span class="linenos">27</span>    <span class="k">def</span> <span class="nf">utility</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">agent</span><span class="p">:</span> <span class="n">RPSAgent</span><span class="p">):</span>
<span class="linenos">28</span>        <span class="k">return</span> <span class="n">agent</span><span class="o">.</span><span class="n">accumulated_payoff</span>
</pre></div>
</div>
</div>
<p>As shown, there are three functions defined in the <code class="docutils literal notranslate"><span class="pre">RPSTrainer</span></code> class.</p>
<ul class="simple">
<li><p>First, in the <code class="docutils literal notranslate"><span class="pre">setup</span></code> function, by using the <code class="docutils literal notranslate"><span class="pre">add_agent_training_property</span></code> function, you can define for which agent list (Line 9), which behavioral parameters (Line 10-14), and which agents in the list (Line 15) you want to train.</p></li>
<li><p>Second, in the <code class="docutils literal notranslate"><span class="pre">collect_data</span></code> function, you can define which results you want to collect, so that you can analyze their evolution processes during the training afterward.</p></li>
<li><p>Third, in the <code class="docutils literal notranslate"><span class="pre">utility</span></code> function, you can define the “target” that the agents want to maximize by updating their behavioral parameters in the evolutionary training process. Please note that, the “target” can be updated every period, but the agents maximize its value in the last period.</p></li>
</ul>
<p>For further details about the <code class="docutils literal notranslate"><span class="pre">Trainer</span></code> module,
please read the <a class="reference external" href="https://ieeexplore.ieee.org/document/9857838/">paper</a> mentioned above.</p>
</section>
<section id="results">
<h2>Results<a class="headerlink" href="#results" title="Permalink to this heading"></a></h2>
<p>After running the <code class="docutils literal notranslate"><span class="pre">run_trainer</span></code> file, four tables will be generated:</p>
<ul class="simple">
<li><p>agents_trainer_result</p></li>
<li><p>agents_trainer_result_cov</p></li>
<li><p>environment_trainer_result</p></li>
<li><p>environment_trainer_result_cov</p></li>
</ul>
<p>As indicated by their names,
the tables contain the evolution processes of the variables that are added in the <code class="docutils literal notranslate"><span class="pre">RPSTrainer.collect_data</span></code> function,
including micro-level variables of the agents and macro-level variables of the environment.
Similar to the <a class="reference internal" href="covid_contagion_calibrator.html#covidcontagioncalibrator"><span class="std std-ref">CovidContagionCalibrator</span></a> model, the “_cov” part in the table name indicates
if the results of the chromosomes are aggregated as “mean” and “coefficient of variance” values.</p>
<p>In this document, we provide the evolution process of one key result at the macro-level:
the total accumulated payoff of all the agents by the end of the simulation.
From the <code class="docutils literal notranslate"><span class="pre">environment_trainer_result_cov</span></code> table, we provide its mean and cov values as below.</p>
<img alt="../_images/trainer_payoff.png" src="../_images/trainer_payoff.png" />
<img alt="../_images/trainer_payoff_cov.png" src="../_images/trainer_payoff_cov.png" />
<p>As shown, in the training generations, the agents update their behavioral parameters so that their payoffs are improved.
Because the agents have “heterogeneous preference”, they are not playing a zero-sum game.
On the other hand, the coefficient of variance (<span class="math notranslate nohighlight">\(\mu / \sigma\)</span>) also declines in the training,
which indicates a convergence, i.e., the stability of this individual GA-based training algorithm.
We tested with two paths and they followed similar pattern.</p>
<p>We believe that there must be also interesting results at the agent-level in the training process.
Feel free to run the model and explore the database.</p>
<p>For more details of the <code class="docutils literal notranslate"><span class="pre">Trainer</span></code> module, please refer to the <a class="reference internal" href="../api/_index.html#api-reference"><span class="std std-ref">API Reference</span></a> section.</p>
</section>
</section>


           </div>
          </div>
          <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
        <a href="covid_contagion_calibrator.html" class="btn btn-neutral float-left" title="CovidContagionCalibrator" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
        <a href="covid_contagion_visual.html" class="btn btn-neutral float-right" title="CovidContagionVisual" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
    </div>

  <hr/>

  <div role="contentinfo">
    <p>&#169; Copyright 2021-2022, ABM4ALL.</p>
  </div>

  Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
    <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
    provided by <a href="https://readthedocs.org">Read the Docs</a>.
   

</footer>
        </div>
      </div>
    </section>
  </div>
  <script>
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script> 

</body>
</html>